Dual lens digital zoom

ABSTRACT

A camera with a pair of lens/sensor combinations, the two lenses having different focal lengths, so that the image from one of the combinations has a field of view approximately two to three times greater than the image from the other combination. As a user of the camera requests a given amount of zoom, the zoomed image provided will come from the lens/sensor combination having the field of view that is next larger than the requested field of view. Thus, if the requested field of view is less than the smaller field of view combination, the zoomed image will be created from the image captured by that combination, using cropping and interpolation if necessary. Similarly, if the requested field of view is greater than the smaller field of view combination, the zoomed image will be created from the image captured by the other combination, using cropping and interpolation if necessary.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/435,080 (now U.S. Pat. No. 8,553,106) entitled “Duel Lens DigitalZoom” filed May 4, 2009 by the same inventors, which is incorporatedherein by reference in its entirety.

BACKGROUND

Digital camera modules are currently being incorporated into a varietyof host devices. Such host devices include cellular telephones, personaldata assistants (PDAs), computers, and so forth. Consumer demand fordigital camera modules in host devices continues to grow.

Host device manufacturers prefer digital camera modules to be small, sothat they can be incorporated into the host device without increasingthe overall size of the host device. Further, there is an increasingdemand for cameras in host devices to have higher-performancecharacteristics. One such characteristic that many higher-performancecameras (e.g., standalone digital still cameras) have is the ability tovary the focal length of the camera to increase and decrease themagnification of the image, typically accomplished with a zoom lens, nowknown as optical zooming. Optical zooming is typically accomplished bymechanically moving lens elements relative to each other, and thus suchzoom lens are typically more expensive, larger, and less reliable thanfixed focal length lenses. An alternative approach for approximatingthis zoom effect is achieved with what is known as digital zooming. Withdigital zooming, instead of varying the focal length of the lens, aprocessor in the camera crops the image and interpolates between thepixels of the captured image to create a “magnified” butlower-resolution image.

There have been some attempts to use two different lenses to approximatethe effect of a zoom lens. It has been done in the past with filmcameras in which the user could select one of two different focallengths to capture an image on film. More recently, a variation on thisconcept with camera modules has been disclosed in U.S. Pat. Pub. No.2008/0030592, the entire contents of which are incorporated herein byreference, which discusses a camera module with a pair of sensors, eachhaving a separate lens through which light is directed to the respectivesensor. In this publication, the two sensors are operated simultaneouslyto capture an image. The respective lenses have different focal lengths,so even though each lens/sensor combination is aligned to look in thesame direction, each will capture an image of the same subject but withtwo different fields of view. The images are then stitched together toform a composite image, with the central portion of the composite imagebeing formed by the relatively higher-resolution image taken by thelens/sensor combination with the longer focal length and the peripheralportion of the composite image being formed by a peripheral portion ofthe relatively lower-resolution image taken by the lens/sensorcombination with the shorter focal length. The user selects a desiredamount of zoom and the composite image is used to interpolate valuestherefrom to provide an image with the desired amount of zoom.Unfortunately, the disclosure in this publication is largely conceptualand lacks in certain details that would be needed to provide optimalperformance. U.S. Pat. App. No. 61/161,621, the entire contents of whichare incorporated herein by reference, discloses improvements andrefinements to this concept.

The foregoing examples of the related art and limitations relatedtherewith are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the drawings.

SUMMARY

Disclosed herein is a camera operated by a user that includes a firstsensor that captures a first image; a first lens that directs light tothe first sensor, the first lens having a first focal length, whereinthe combination of the first sensor and first lens has a first field ofview; a second sensor that captures a second image; a second lens thatdirects light to the second sensor, the second lens having a secondfocal length that is longer than the first focal length, wherein thecombination of the second sensor and second lens has a second field ofview, wherein the first field of view is greater than the second fieldof view; and a zoom control operable by the user to allow the user torequest a desired field of view to produce a zoomed image. Thecombination of the first sensor and the first lens are substantiallyaligned with the combination of the second sensor and the second lens toallow each to be directed toward the same subject. The zoomed image isthe first image if the requested field of view is substantially equal tothe first field of view, the zoomed image is produced from the firstimage by cropping and interpolating the first image if the requestedfield of view is less than the first field of view and greater than thesecond field of view, the zoomed image is the second image if therequested field of view is substantially equal to the second field ofview, and the zoomed image is produced from the second image by croppingand interpolating the second image if the requested field of view isless than the second field of view.

The camera may further include a third sensor that captures a thirdimage; a third lens that directs light to the third sensor, the thirdlens having a third focal length that is longer than the second focallength, wherein the combination of the third sensor and third lens has athird field of view, wherein the second field of view is greater thanthe third field of view. The combination of the third sensor and thethird lens may be substantially aligned with the combination of thefirst sensor and the first lens and the combination of the second sensorand the second lens to allow each to be directed toward the samesubject. The zoomed image may be the first image if the requested fieldof view is substantially equal to the first field of view, the zoomedimage may be produced from the first image by cropping and interpolatingthe first image if the requested field of view is less than the firstfield of view and greater than the second field of view, the zoomedimage may be the second image if the requested field of view issubstantially equal to the second field of view, the zoomed image may beproduced from the second image by cropping and interpolating the secondimage if the requested field of view is less than the second field ofview and greater than the third field of view, the zoomed image may bethe third image if the requested field of view is substantially equal tothe third field of view, and the zoomed image may be produced from thethird image by cropping and interpolating the third image if therequested field of view is less than the third field of view.

The first field of view may be approximately twice that of the secondfield of view. The first field of view may be in the range ofapproximately two to three times that of the second field of view. Thezoom control may be used to request a zoomed image in a range, with oneend of the range corresponding approximately to the first field of viewand the opposite end of the range corresponding approximately to ¼ to ½of the second field of view.

Also disclosed is a camera operated by a user that includes a sensorthat captures an image; a first lens that can direct light to thesensor, the first lens having a first focal length, wherein thecombination of the sensor and first lens has a first field of view; asecond lens that can direct light to the sensor, the second lens havinga second focal length that is longer than the first focal length,wherein the combination of the sensor and second lens has a second fieldof view, wherein the first field of view is greater than the secondfield of view; and a zoom control operable by the user to allow the userto request a desired field of view to produce a zoomed image. The firstand second lenses can be moved relative to the sensor into one of twodifferent positions so that the sensor can receive either light passingthrough the first lens or light passing through the second lens. Thecombination of the sensor and the first lens are substantially alignedwith the combination of the sensor and the second lens to allow eitherto be directed toward the same subject. The zoomed image is the imagefrom the first sensor if the requested field of view is substantiallyequal to the first field of view, the zoomed image is produced from theimage from the first sensor by cropping and interpolating the image fromthe first sensor if the requested field of view is less than the firstfield of view and greater than the second field of view, the zoomedimage is the image from the second sensor if the requested field of viewis substantially equal to the second field of view, and the zoomed imageis produced from the image from the second sensor by cropping andinterpolating the image from the second sensor if the requested field ofview is less than the second field of view.

The first field of view may be approximately twice that of the secondfield of view. The first field of view may be in the range ofapproximately two to three times that of the second field of view. Thezoom control may be used to request a zoomed image in a range, with oneend of the range corresponding approximately to the first field of viewand the opposite end of the range corresponding approximately to ¼ to ½of the second field of view. The lenses may be moved relative to thesensor manually by the user. The lenses may be moved relative to thesensor automatically by the camera when the field of view requested bythe user changes between using the image from one of the combinations tothe other of the combinations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a camera.

DETAILED DESCRIPTION

The following description is not intended to limit the invention to theform disclosed herein. Consequently, variations and modificationscommensurate with the following teachings, and skill and knowledge ofthe relevant art, are within the scope of the present invention. Theembodiments described herein are further intended to explain modes knownof practicing the invention and to enable others skilled in the art toutilize the invention in such, or other embodiments and with variousmodifications required by the particular application(s) or use(s) of thepresent invention.

A camera 10 is shown in FIG. 1. The camera 10 may include a first lens12 having a relatively-shorter focal length and a first sensor 14 thatare located proximate to and substantially aligned with a second lens 16having a relatively-longer focal length and a second sensor 18. Byhaving the combined first lens and first sensor aligned with thecombined second lens and second sensor, the sensors can each obtain animage of substantially the same subject. Of course, due to the differentfocal lengths of the lenses 12 and 16, the first sensor 14 will obtainan image of the subject with a relatively-wider field of view (FOV) ascompared to the relatively-narrower FOV of the image obtained by thesecond sensor 18. In one example, the first FOV may be in the range oftwo to three times as large as the second FOV. In another example, thefirst FOV may be approximately twice the second FOV.

In most cases, each sensor 14 and 18 may perform certain basic imageprocessing algorithms such as white balancing, and so forth. The lenses12 and 16 could be made of any acceptable material, including plastic(e.g., injection-molded plastic), glass, optical ceramic, diffractiveelements, or a composite.

In one example, the lens 16 may be a lens having a focal length of 7.2mm and a field-of-view (FOV) of 32 degrees, while the lens 12 may be alens having a focal length of 3.62 mm and an FOV of 63 degrees. Theselens specifications are merely exemplary and any other suitable lenscharacteristics could be acceptable. In addition, one or both of thelenses 12 and 16 could be variable focal length (zoom) lenses.

In one example, the two lenses 12 and 16 may have the same f-number sothat the illuminance of the light received at the sensors 14 and 18 isequivalent. With equivalent illuminance, the sensors can be operated atsimilar levels of amplification and with similar exposure times. In thismanner, the separate images captured by the separate sensors 14 and 18can be of similar levels of brightness and contrast. By having similarlevels of amplification, the background noise in each image will besimilar. By having similar exposure times, artifacts in each image dueto subject motion will be similar. By maintaining similarity as to thesetwo characteristics in the two images, transitions between the twoimages will be more acceptable to the user. In another example, thelenses 12 and 16 may be chosen to provide the same depth of field foreach lens/sensor combination.

In one example, each of the sensors is a Bayer sensor, which uses acolor filter array over the array of separate pixels, as is well known.Such sensors sense green light at every other pixel, with theintervening pixels alternating between red pixels and blue pixels. Theraw sensed signals are later provided to a demosaicing algorithm, whichinterpolates between the pixels to obtain a full color signal for eachpixel. However, the invention is not limited to use with a Bayer sensorand will work equally well with sensors having a different color filterarray, cameras based on time-sequential color, cameras usingbeamsplitters and separate sensors for each color channel, and othercamera architectures.

In some cases, the camera 10 may be considered to include only thefunctional portions described above. In other cases, these portions(referred to collectively as a camera module 22) may also be combinedwith certain downstream components as part of the camera 10. In suchcase, the camera 10 may also include an image signal processor (ISP) 24,a display 26, and user interface controls 28. Of course, as is wellknown in the camera industry, cameras may also typically include severalother components that are omitted here for simplification. For example,as non-limiting examples, these other components may include batteries,power supplies, an interface for the application of external power, aUSB or other interface to a computer and/or printer, a light source forflash photography, auto-focus and image stability controls, internalmemory, one or more ports for receiving an external memory card ordevice (e.g., an SD or xD memory card), and in the case of the use of acamera in a mobile phone, a microphone, speaker, transmitter/receiver,and an interface for an external microphone and speaker (e.g., aBluetooth headset).

The user interface controls 28 may include conventional controls thatare used to operate the camera, including controls to instruct thecamera to capture one or more images, as well as to manipulate theimages, and many other functions. One of the controls allows the user todigitally zoom the camera to increase or decrease the field of view(FOV) of the camera. The user can zoom the image out to the FOV of theimage from the first sensor 14 at one end of the zooming range and to apoint that may be somewhere between ½ and ¼ of the FOV of the image fromthe second sensor 18 at the other end of the zooming range. The zoomedimage may simply be the first image if the requested field of view issubstantially equal to the first field of view. The zoomed image isproduced from the first image by cropping and interpolating the firstimage if the requested field of view is less than the first field ofview and greater than the second field of view. The zoomed image is thesecond image if the requested field of view is substantially equal tothe second field of view. The zoomed image is produced from the secondimage by cropping and interpolating the second image if the requestedfield of view is less than the second field of view.

The zooming range may be limited on one end by the FOV of the firstsensor. Although the camera 10 could be designed to allow the user tocontinue to zoom out to a “FOV” greater than that of the firstlens/sensor, the image would in fact get smaller as the image from thefirst sensor was merely shrunk in size and no extra image informationwould be brought at the margins of the image, because there is no widerFOV image data to use. The zooming range may be limited on the oppositeend by the amount of digital zooming that is deemed to be acceptable tousers. Due to the image interpolation that occurs, it may be desirableto limit the digital zooming to a FOV that is somewhere between ½ and ¼of the FOV of the image from the second sensor 18.

Alternatively, the camera module 22 could include one or more ISPslocated thereon. They could be separate from or integrated into thesensors. Further, while the lenses 12 and 16 described herein are fixedfocal length, either or both could be variable focal length (zoom)lenses.

Alternatively, the camera 10 could be provided with a third lens/sensorcombination that is aligned with the first two lens/sensor combinations.This third lens/sensor combination may have a field of view that isstill smaller than that of the second lens/sensor combination. As thecamera was zoomed by the user, the zoomed image would transition betweenbeing formed from the first image, the second image, and the third imagein a similar manner to that described above. In such case, the zoomrange might be from the field of view of the first lens/sensorcombination to ½ to ¼ of the field of view of the third lens/sensorcombination.

As another alternative, there may be only one sensor and the two (ormore) lenses may be moved relative to the sensor to allow an image to becaptured from either the combination of the first lens and the onesensor or from the combination of the second lens and the one sensor.This relative movement of the lenses and sensor could be achieved by theuser sliding or actuating some type of mechanical member associatedtherewith or it could be achieved by the camera automatically moving thelenses relative to the sensor when the requested amount of zoomingcauses the zoomed image to switch from coming from one combination tocoming from the other combination. As a further variation to all ofthis, instead of moving the lenses or sensor, the light path could beredirected by a mirror or the like to cause light from a selected lensto impinge upon the sensor.

Any other combination of all the techniques discussed herein is alsopossible. The foregoing description has been presented for purposes ofillustration and description. Furthermore, the description is notintended to limit the invention to the form disclosed herein. While anumber of exemplary aspects and embodiments have been discussed above,those of skill in the art will recognize certain variations,modifications, permutations, additions, and sub-combinations thereof. Itis therefore intended that the following appended claims and claimshereafter introduced are interpreted to include all such variations,modifications, permutations, additions, and sub-combinations as arewithin their true spirit and scope.

I claim:
 1. A camera operated by a user, comprising: a sensor thatcaptures an image; a first lens that can direct light to the sensor, thefirst lens having a first focal length, wherein the combination of thesensor and first lens has a first field of view; a second lens that candirect light to the sensor, the second lens having a second focal lengththat is shorter than the first focal length, wherein the combination ofthe sensor and second lens has a second field of view, wherein thecombination of the first field of view and the second field of viewcomprises a field of view that is greater than the first field of view;and a zoom control operable by the user to allow the user to request adesired field of view to produce a zoomed image; and wherein the firstand second lenses can be moved into and out of optical alignment withthe sensor so that the sensor receives light passing through the firstlens including the first field of view when the first lens is alignedwith the sensor and light passing through the second lens including thesecond field of view when the second lens is aligned with the sensor atvarious focal distances; wherein the combination of the sensor and thefirst lens are aligned with the combination of the sensor and the secondlens to allow the camera to focus from a same focus distance at saidfirst field of view through said first lens according to said firstfocal length and at said second field of view through said second lensaccording to said second focal length, respectively; wherein the zoomedimage comprises the image focused to the sensor from the first lens ifthe requested field of view is equal to the first field of view, thezoomed image is produced from the image focused to the sensor from thefirst lens by cropping and interpolating the image focused to the sensorfrom the first lens if the requested field of view is within but lessthan the first field of view, and wherein the zoomed image comprises theimage focused to the sensor from the second lens if the requested fieldof view is equal to the second field of view, and the zoomed image isproduced from the image focused to the sensor from the second lens bycropping and interpolating the image focused to the sensor from thesecond lens if the requested field of view is within but less than thesecond field of view and greater than the first field of view; andwherein the depth of field for the combination of the sensor and thefirst lens is the same as the depth of field for the combination of thesensor and the second lens.
 2. A camera as defined in claim 1, whereinthe second field of view is twice that of the first field of view.
 3. Acamera as defined in claim 1, wherein the second field of view is in therange of two to three times that of the first field of view.
 4. A cameraas defined in claim 1, wherein the zoom control can be used to requestthe zoomed image in a range, with one end of the range corresponding tothe second field of view and the opposite end of the range correspondingto ¼ to ½ of the first field of view.
 5. A camera as defined in claim 1,wherein the first and second lenses are moved into and out of opticalalignment with the sensor automatically by the camera when the field ofview requested by the user changes between using the image from one ofthe combinations to the other of the combinations.
 6. A camera operatedby a user, comprising: a sensor that captures an image; a first lensthat can direct light to the sensor, the first lens having a first focallength, wherein the combination of the sensor and first lens has a firstfield of view; a second lens that can direct light to the sensor, thesecond lens having a second focal length that is shorter than the firstfocal length, wherein the combination of the sensor and second lens hasa second field of view, wherein the combination of the first field ofview and the second field of view comprises a field of view that isgreater than the first field of view; and a zoom control operable by theuser to allow the user to request a desired field of view to produce azoomed image; and wherein the first and second lenses can be movedrelative to the sensor into and out of one of multiple differentpositions where the sensor receives light passing through the first lensincluding the first field of view when the first lens is in the one ofthe multiple positions and light passing through the second lensincluding the second field of view when the second lens is in the one ofthe multiple positions at various focal distances; wherein thecombination of the sensor and the first lens are aligned with thecombination of the sensor and the second lens to allow the camera tofocus from a same focus distance at said first field of view throughsaid first lens according to said first focal length and at said secondfield of view through said second lens according to said second focallength, respectively; wherein the zoomed image comprises the imagefocused to the sensor from the first lens if the requested field of viewis equal to the first field of view, the zoomed image is produced fromthe image focused to the sensor from the first lens by cropping andinterpolating the image focused to the sensor from the first lens if therequested field of view is within but less than the first field of view,and wherein the zoomed image comprises the image focused to the sensorfrom the second lens if the requested field of view is equal to thesecond field of view, and the zoomed image is produced from the imagefocused to the sensor from the second lens by cropping and interpolatingthe image focused to the sensor from the second lens if the requestedfield of view is within but less than the second field of view andgreater than the first field of view; and wherein the depth of field forthe combination of the sensor and the first lens is the same as thedepth of field for the combination of the sensor and the second lens. 7.A camera as defined in claim 6, wherein the second field of view istwice that of the first field of view.
 8. A camera as defined in claim6, wherein the second field of view is in the range of two to threetimes that of the first field of view.
 9. A camera as defined in claim6, wherein the zoom control can be used to request the zoomed image in arange, with one end of the range corresponding to the second field ofview and the opposite end of the range corresponding to ¼ to ½ of thefirst field of view.
 10. A camera as defined in claim 6, wherein thefirst and second lenses are moved relative to the sensor automaticallyby the camera when the field of view requested by the user changesbetween using the image from one of the combinations to the other of thecombinations.
 11. In a camera operated by a user, a method comprising:capturing an image with a sensor; directing light to the sensor with afirst lens, the first lens having a first focal length, wherein thecombination of the sensor and first lens has a first field of view;directing light to the sensor with a second lens, the second lens havinga second focal length that is shorter than the first focal length,wherein the combination of the sensor and second lens has a second fieldof view, wherein the combination of the first field of view and thesecond field of view comprises a field of view that is greater than thefirst field of view; and using a zoom control operable by the user toallow the user to request a desired field of view to produce a zoomedimage; and wherein the first and second lenses can be moved into and outof optical alignment with the sensor so that the sensor receives lightpassing through the first lens including the first field of view whenthe first lens is aligned with the sensor and light passing through thesecond lens including the second field of view when the second lens isaligned with the sensor at various focal distances; wherein thecombination of the sensor and the first lens are aligned with thecombination of the sensor and the second lens to allow the camera tofocus from a same focus distance at said first field of view throughsaid first lens according to said first focal length and at said secondfield of view through said second lens according to said second focallength, respectively; wherein the zoomed image comprises the imagefocused to the sensor from the first lens if the requested field of viewis equal to the first field of view, the zoomed image is produced fromthe image focused to the sensor from the first lens by cropping andinterpolating the image focused to the sensor from the first lens if therequested field of view is within but less than the first field of view,and wherein the zoomed image comprises the image focused to the sensorfrom the second lens if the requested field of view is equal to thesecond field of view, and the zoomed image is produced from the imagefocused to the sensor from the second lens by cropping and interpolatingthe image focused to the sensor from the second lens if the requestedfield of view is within but less than the second field of view andgreater than the first field of view; and wherein the depth of field forthe combination of the sensor and the first lens is the same as thedepth of field for the combination of the sensor and the second lens.12. A method as defined in claim 11, wherein the second field of view istwice that of the first field of view.
 13. A method as defined in claim11, wherein the second field of view is in the range of two to threetimes that of the first field of view.
 14. A method as defined in claim11, wherein the zoom control can be used to request the zoomed image ina range, with one end of the range corresponding to the second field ofview and the opposite end of the range corresponding to ¼ to ½ of thefirst field of view.
 15. A method as defined in claim 11, wherein thefirst and second lenses are moved into and out of optical alignment withthe sensor automatically by the camera when the field of view requestedby the user changes between using the image from one of the combinationsto the other of the combinations.
 16. In a camera operated by a user, amethod comprising: capturing an image with a sensor; directing light tothe sensor with a first lens, the first lens having a first focallength, wherein the combination of the sensor and first lens has a firstfield of view; directing light to the sensor with a second lens, thesecond lens having a second focal length that is shorter than the firstfocal length, wherein the combination of the sensor and second lens hasa second field of view, wherein the combination of the first field ofview and the second field of view comprises a field of view that isgreater than the first field of view; and using a zoom control operableby the user to allow the user to request a desired field of view toproduce a zoomed image; and wherein the first and second lenses can bemoved relative to the sensor into and out of one of multiple differentpositions where the sensor receives light passing through the first lensincluding the first field of view when the first lens is in the one ofthe multiple positions and light passing through the second lensincluding the second field of view when the second lens is in the one ofthe multiple positions at various focal distances; wherein thecombination of the sensor and the first lens are aligned with thecombination of the sensor and the second lens to allow the camera tofocus from a same focus distance at said first field of view throughsaid first lens according to said first focal length and at said secondfield of view through said second lens according to said second focallength, respectively; wherein the zoomed image comprises the imagefocused to the sensor from the first lens if the requested field of viewis equal to the first field of view, the zoomed image is produced fromthe image focused to the sensor from the first lens by cropping andinterpolating the image focused to the sensor from the first lens if therequested field of view is within but less than the first field of view,and wherein the zoomed image comprises the image focused to the sensorfrom the second lens if the requested field of view is equal to thesecond field of view, and the zoomed image is produced from the imagefocused to the sensor from the second lens by cropping and interpolatingthe image focused to the sensor from the second lens if the requestedfield of view is within but less than the second field of view andgreater than the first field of view; and wherein the depth of field forthe combination of the sensor and the first lens is the same as thedepth of field for the combination of the sensor and the second lens.17. A method as defined in claim 16, wherein the second field of view istwice that of the first field of view.
 18. A method as defined in claim16, wherein the second field of view is in the range of two to threetimes that of the first field of view.
 19. A method as defined in claim16, wherein the zoom control can be used to request the zoomed image ina range, with one end of the range corresponding to the second field ofview and the opposite end of the range corresponding to ¼ to ½ of thefirst field of view.
 20. A method as defined in claim 16, wherein thefirst and second lenses are moved relative to the sensor automaticallyby the camera when the field of view requested by the user changesbetween using the image from one of the combinations to the other of thecombinations.